Balls into Bins with Strict Capacities and Weighted 

Edges 



Ankur Sahai 
No Institute Given 



Abstract. We explore a novel theoretical model for studying the performance 
of distributed storage management systems where the data-centers have limited 
capacities (as compared to storage space requested by the users). Prior schemes 
such as Balls-into-bins (used for load balancing) neither consider bin (consumer) 
capacities (multiple balls into a bin) nor the future performance of the system 
after, balls (producer requests) are allocated to bins and restrict number of balls 
as a function of the number of bins. Our problem consists of finding an optimal 
assignment of the online producer requests to consumers (via weighted edges) in 
a complete bipartite graph while ensuring that the total size of request assigned 
on a consumer is limited by its capacity. The metric used to measure the per- 
formance in this model is the (minimization of) weighted sum of the requests 
assigned on the edges (loads) and their corresponding weights. We first explore 
the optimal offline algorithms followed by the analysis of different online tech- 
niques (by comparing their performance against the optimal offline solution). LP 
and Primal-Dual algorithms are used for calculating the optimal offline solution 
in 0(r ■ n) time (where r and n are the number of requests and consumers respec- 
tively) while randomized algorithms are used for the online case. 

We propose randomized online algorithms in which the consumers are selected 
based on edge probabilities (that can change with consumer failures; due to ca- 
pacity exhaustion) and evaluate the performance of these randomized schemes 
using probabilistic analysis. The performance of the online algorithms is mea- 
sured using competitive analysis assuming an oblivious adversary who knows 
the randomized algorithm but not the results produced. For the simplified model 
with equal consumer capacities an average-case competitive ratio (which com- 
pares the average cost of the output produced by the online algorithm and the 
minimum cost of the optimal offline solution) of ( min d , d , , ) (where d is the edge 
weight / distance) is achieved using an algorithm that has equal probability for 
selecting any of the available edges with a running time of 0(r). In the extending 
the model to arbitrary consumer capacities we show an average case competitive 
ratio of ( - mi ^ c rf ). This theoretical model gives insights to a (storage) cloud sys- 
tem designer about, how the different attributes (producer requests, edge weights 
and consumer capacities) effect the overall (read / write) performance of a dis- 
tributed storage management system over a period of time. 



Keywords: Online algorithms, Primal-dual, Load balancing, Com- 
petitive ratio. 



1 Motivation 



A (storage cloud) is a collection of VMs (represented by consumers in our model in 
section [2]i and data-centers (producers) that interact with each other through network 
links with different bandwidths / data-transfer rates (inversely proportional to the edge 
weights). VMs generate requests for virtual disk space (requests) at the time of creation. 
The data-centers have limited storage space (capacity). One of the core problems is to 
optimally select a data-center for allocating the disk space for a VM. If we assume that 
all data-centers are available for the requests generated by each VM then this config- 
uration forms a complete bipartite graph. As the VM users are added gradually to the 
cloud and their disk-space requirements need to be satisfied instantly to meet the SLAs 
this forms the online part of the problem. We assumes that once a VM is allocated stor- 
age space on a particular data-center it cannot be moved to a different one, thus VMs 
read / write data using the same network links (fixed at the time of VM provisioning) 
throughout their lifetime. 

The overall I/O performance of the cloud storage system can be optimized when 
the quality of network links used for the majority of read / write operations is max- 
imized which, can be acheived by using the higher quality (inversely proportional to 
edge weights) links for allocating as much producer requests as possible. This is equiv- 
alent to minimizing the the distance weighted sum of request allocated in the system. 
This theoretical model can be used to measure the performace of distributed storage 
provisionig schemes such as VMware's virtualization framework - VSphere B8I9I . 

2 Problem Definition 

In the offline model Aiorr the requests sizes s(t), capacities Cj and edge weights dij 
are arbitrary. 

- A complete (undirected) bipartite graph G = (P, C, E). 

- Set P = {Pi | i e [m]} of producers, m = \P\. Set C = {Cj \ j e [n]} of consumers, 
n = \C\, with arbitrary capacity c. The capacity is a strict bound for the consumer's 
load and cannot be exceeded. 

- Set of edges E = {ey | i e [m],j e [n]} connect each producer to each consumer 
and with d U j which denote the distance between producer P, and cj. 

- r is the total number of requests. The requests are produced by the producers and 
allocated to the consumers. A request can be split and allocated to different produc- 
ers. s(t) is the size of the request produced at time t. 

- Producers are selected at random in each round. 

- d{t) is the distance between the producer and consumer that are chosen in time step 
t. The total cost of the random allocation is G = YJt=\ *( f ) ' °^ f )- 

- We assume that every request can commit to at least one consumer (regardless of 
the previous allocation), i. e. at least one consumer must have sufficient space left. 
The number and size of requests must be limited accordingly. 

- (j is the load on consumer cj. All loads t\, ...,(„ are set to in the beginning. 
Whenever a consumer receives a request, its load is increased by the size of the 
request. 
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- £ij is the total request from producer P, allocated to consumer Cj known as the 
edge load. All loads €\ t \, („ hn are set to in the beginning. Whenever a consumer 
receives a request from a producer the corresponding edge load is increased by the 
size of the request. 

The objective of this problem is to find an average-case [2| a-competitive online 
algorithm (assuming startup cost B = 0), that minimizes the expected value of weighted 
sum of edge loads while satisfying the producer requests: 



E 



di,j(t) ■ I, ,11) 



<a-OPT!(t),\/(t,r) (1) 



where, a is a constant and OPTi(t) is the output of the optimal offline algorithm for 
the input received in the time interval (0, t] for an instance I. 

Equation (|2ji guarantees that the total load on the edges ( 2>e[n] U,j(t) ) incident 
on each producer (i e [to]) is equal to the total size of the requests generated by the 
producer (2,^,(0)- This will be referred to as producer request constraint. 

J]kj(t) = J] Ri(t)y(ielml f) (2) 

Equation (|3jl ensures that the total load on the edges (2i€[m] UjO)) incident on a 
consumer (j e [n]) does not exceed the consumer capacity (cj). This will be referred to 
as consumer capacity constraint. 

X'uW<c„V(i6[«],0 (3) 

ie[m] 



3 Related Work 

Balls-into-Bins II 1 14161 model is used for studying load balancing in a similar resource 
allocation configuration where, the objective is to place m balls into n bins while guar- 
anteeing bounds on the maximum, minimum or the average load across all the bins. 
The main advantage of the model defined in section|2]is that it also takes into accounts 
the capacities of the bins (which are analogous to the consumers in the problem defined 
in section [2]). Further the model described in section [2] compares the performance of a 
randomized algorithm with the optimal offline algorithm. 

One of the well studied online algorithm for assigning resources to users is the k- 
server problem [7] where, the servers handle request once for each client. Dynamic 
assignment [3 1 has a similar configuration involving bipartite graphs. 



4 Offline Algorithms 

The optimal offline algorithm has to exhaustively look at the available edges for allo- 
cating a request. This paper uses Linear Programming and Primal-Dual algorithms for 
solving the offline version (section|2]i. 
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4.1 Linear Programming 



Linear Programming J5J is a method used to solve large-scale optimization problems 
with a set of constraints and an objective function (minimization or maximization) both 
being linear. 

The LP formulation for this problem is as follows, 

Objective function: 

Minimize: 

2 d u (t) ■ kj(t), dij(t) > 0, hj(t) > (4) 

ie[m],je[n] 

Constraints: 

2 kj(t) > 2^(0, Ri(f) > 0, V(i e [m], t) (5) 

J] iij(t) < Cj => - 2 W * ~ c h v 0' e M (6) 

ie[m] i€[m] 

This LP is used for calculating the optimal solution OPT(t) for the input received 
in the time interval (0, t], t. As the requests i e [to] are non-negative the load assign- 
ments U j in the objective function ([4]) are also non-negative. Equation |5]) represents 
the producer request constraint corresponding to |2]) whereas |6]) corresponds to the 
consumer capacity constraint corresponding to In addition to this, new constraints 
corresponding to the existing load assignment on edges have to be added at each time 
instance t for assignment without reallocation (defined in section|2]). 

LP formulation in section |4~T| produces a feasible ( without violating the consumer 
capacity constraints ^ ) assignment of loads on edges e,j corresponding to the 
requests R,(t), V(; e [m], t). Equation <|3j guarantees that the total request generated by 
producers i e [m] is satisfied and (|6| ensures that the capacities of consumers j e [n] are 
not exceeded. By definition of problem in section|2] this is a valid assignment of loads 
on edges. LP formulation in section [4T| produces the optimal assignment of loads lij on 
edges eij corresponding to the requests Rj(t), V(; e [to], t). The solution produced by 
LP is optimal as fractional loads are allowed and the LP considers all possible solutions 
minimize the objective function (|4]). 

4.2 Primal-Dual 

Primal-Dual algorithms are used for a certain class of optimization problems where 
there are a finite number of feasible solutions available at each step. The dual is often 
useful for providing intuitions about the nature of the solution that are implicit in the 
primal. 

Consider the dual of the LP formulation in section |4T| Let y, be the dual variables 
corresponding to producers i e [to] |3]l and Zj be the dual variables corresponding to the 
consumers j e [n] |6} then the corresponding dual is, 

Objective function: 

Maximize: 
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X- r ^ v>() - z j-° (7) 

R, = 2, f = i /?i(0 is the total size of the requests produced by P, 
Constraints: 

yi-Zj<d u ,V(i€[m],j€[n]) (8) 

Equation ([H} suggests that the potential difference between producers and con- 
sumers can be atmost equal to d,j. This will be referred to as dual potential limit con- 
straint. 

According to complementary slackness conditions, 

l u > <=> y, , - zj = dij (9) 

Let T be the set of tight constraints (which represent edges selected by Algorithm 
|TJ> (7" = {(/, j) | yi - Zj — dij, V(; G [m],j e [n])}) and S, the set of slack constraints (S = 
{(*> j) I _ Zj < dij, V(; G [m], j G [n])}). Then consider an unit benefit function which 
measures the increase in dual objective (|7|l corresponding to a unit amount of request 
{Ay, ■ Ri(t)) produced by producer i G [m] at time t where, Azj are the corresponding 
increases in the consumer-dual variables; required for keeping the dual potential limit 
constraints ([H} tight. 



Bi(t) = A Tl 



(10) 



The unit benefit function in 10 is used as a criteria for selecting the request which 
produces the maximum increase in the dual objective function by the Primal-dual algo- 
rithm. 



Algorithm 1 Primal-Dual algorithm for Offline Assignment 

1: T <- 

2: S<-{«,J)|V(iG[m],jG[»])} 

3: y t <- 0,Vi 6 [m] 

4: Zj «— 0, Vj 6 [n] 

5: iy«-0,V(iEM,jE[/i]) 

6: while 3(i, t) : B,(f) > do 

7: y, : max, (B,-(f) j 

8: : mini,/. - Cy ( - z,) | i e [m], j e [n]} 

9: I; </.-(y,-;.. 

10: •y i «-y,+z)i 

11: zj*-zj+A u VjeT 

12: T = TU (i, j) 

13: S = S\ (!,./) 

14: ^ 2 = Cj - 2(,j)er 2y 

15: /,j «- Ifj +A 2 

16: end while 
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The primal-dual Algorithm [T] chooses the (producer dual variable y, correspond- 
ing to) request /?,(?) with the highest benefit function B,{t) (step 7 in Algorithm [T} to 
maximize the increase in value of dual objective function and then chooses the dual po- 
tential limit constraint (operation 12) (and the corresponding variable zj) that is closest 
to becoming tight at step 8 in Algorithm [T] (corresponding to the least cost edge in the 
primal) and increases the value of y, by the amount (A\, in steps 9 and 10 in Algorithm 
[l} that is needed to make this constraint tight (which corresponds to selecting an edge 
for allocating a request in the primal using complementary slackness condition [9j. If 
this constraint is tight (T updated in step 12) then corresponding Zj are also increased 
by A i (step 1 1 in Algorithm[T]i to maintain tightness. 

Increasing dij by amount A<x is only symbolic / superficial. The practical output of 
the algorithm can be traced by looking up the set of dual constraints (corresponding 
to edges in primal) that are selected corresponding to each request. Intuitively, optimal 
offline primal-dual Algorithm[T]assigns the lower sized requests to the higher cost edges 
and uses the lower cost edges for the higher sized requests. 

Theorem 1 (Optimality of Algorithm [1}. The Primal-Dual Algorithm [7] reaches the 
optimal solution for the assignment without reallocation problem in section^when it is 
not possible to increase the cost of the dual objective function 

Proof, nitializing y,- <— 0, Vz e [m] and Zj <— 0, Vy e [n] (steps 1 and 2 in Algorithm 
[TJ and from the way we increase the dual variables (step 9 in Algorithm [T| the dual 
potential limit constraint ^ is always satisfied. Setting l, j <— 0,V(i 6 [m],j e [«]) 
(step 5 in Algorithm [TJ makes sure that the primal is feasible at the start. The primal 
consumer capacity constraints in (|6jl are not violated by the way we increase l,j from 
steps 14 and 15 in Algorithm[T] 

When the cost of the dual objective function |7} cannot be increased further, 

Y J Ay i -R i -Y J ^i-Cj = V (ID 

ieT jeT 

=> Yj AyrRi = Yj Az i' c i (12) 

ieT jeT 

At this point, if there was a pending request /?,■(?) with a positive unit benefit function 
that could choose one of the slack (S) dual constraints ( without violating any of the 
potential limit dual constraints!^ ) then the corresponding producer dual variable y; 
can be increased until the dual constraint became tight. By contradiction such a request 
does not exists by definition (step 6) of Algorithm [T] otherwise, the algorithm would 
have continued. So the system is in equilibrium and changing the value of any of the 
dual variables will violate (one or more) dual potential limit constraints. 

However, for the current solution ((y,-, zj) e T), the dual variables need to be changed 
by an equal amount (Ay t = Azi = A, V(/ € [m],j € [n])) for maintaining the potential 
equilibrium [keeping the dual constraints in T tight for meeting complementary slack- 
ness |9| condition]. 

A yi =Azj=A (13) 
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When it is not possible to increase the value of dual objective function then using 
(12) and (13), 



Z*<=2> (14) 

ieT jeT 

From steps 14 and 15 in Algorithm [T] we know that, 

(15) 



Using ( 14 1 and ( 15 i we infer that the requests have been met, 

2*i = Z / w ( ' ) (16) 

This means that the primal producer request constraints in |5]l are feasible and the 
primal consumer capacity constraints in d6]l and dual potential limit constraints are 
always feasible. Complementary slackness d9| is satisfied as we only increase dj j when 
the dual constraint is tight. This means that the primal is optimal and the dual is optimal. 
Thus, this is the optimal solution for the problem in section|2] 

Theorem 2 (Time complexity of Algorithm [!}. The Primal-Dual Algorithm [7] takes 
0(r ■ n), time to complete where, d and n the number of requests and consumers respec- 
tively. 

Proof. At each step of the Algorithm [T](9(r) (where r is the number of requests) oper- 
ations are needed to select the (producer dual variable y, corresponding to the) request 
with maximum unit benefit function, another 0(n) operations to find the corresponding 
consumer dual variable Zj. Hence, it takes 0(d + ri) operations for the first (while loop 
from steps 6 to 16 in Algorithm [TJ and 0(d — l+ n) operations for the second request 
by keeping track of the requests that have been covered by setting a flag in a hash table. 
This gives a time complexity of 0(r ■ n) using amortized analysis. 



5 Online Algorithms 

In the beginning we look at the following simplified model At, 

- Let R(t) be the request in round t and let C(t) denote the set of consumers in round 
t that have enough space left to store request R(t). 

- All requests R(t) have the same size s. 

- All consumers C, have equal capacities c. 

- An oblivious online adversary who can only manipulate the size of the online re- 
quests. 

We use randomized algorithms for the online version as for a deterministic algo- 
rithm the oblivious online adversary who knows the edge selected in each round can 
manipulate the online request sizes to distort the value of the objective function ([TJ. 

The simplified algorithm M is: 



7 



Algorithm 2 Online Algorithm M 

1: Given: P; C; c; d^fi s 

2: Initialize: €j = Vy € [»]; G = 0; « = |C| 

3: for / = 1 to r do 

4: Choose P t e [m] independently and uniformly at random 
5: C(f) = {cj | y 6 [n] Ae j + s<c} 

6: Choose c,- e [n] independently and uniformly at random from C(t). 
7: I j = tj + s 
8: G = G + s- d it j 

9: end for 



Theorem 3. Assuming model M and running algorithm 3\, the expected total cost is 
E[G] = Z r r=1 * ■ d. 

Proof. Using the law of total probability over the different combination of consumer 
failures |C,| - n-k when there are n - k consumers available, 



p(e i=PiJ | (\C(t)\ =n-k)) = Y j P (e i=PiJ \ \Cj =n-k)- p(\Cj\ =n-k) (17) 



n- 1 



(18) 



(19) 



Using ( 19 1 and the law of total expectation over the number of consumers available, 



E[d i=Puj | P] = J /-. | ( /, , | (|C(?)I = « - ^)] • p(|C(0l = « - k) (20) 

= J= ■ Y J p(\C{t)\=n-k) (21) 

= 1 (22) 
n 

We calculate the expected distance of an edge selected in any round over the sce- 
narios corresponding to different number k of consumers available using (21). As the 
probability to select an edge does not change in the different scenarios we deduce in 
(18), (19) and (20) that, there is uniform probability of selecting any (consumer) edge 
given that producer P, is selected in a round. Intuitively this follows form symmetry as 
each consumer is equally likely to be picked in any round assuming equal capacities. 

Using the law of total expectation over the producer selected in a round, 
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E[dij] = J E[d i=PtJ | i = P t ] ■ p(i = Pi) (23) 

i=l 

"' r1 " n 1 

xim<i : 



i=l 7=1 

m n 

-££4u=5 (25) 



m .. 

<=1 7=1 



So the expected cost of Algorithm J[ ( ij) in any round is E[G] = s-.E[djj] = s-d The 
best possible cost of the optimal offline a gorithm OPT in any round is when it chooses 
the cheapest edge miny dij in each round. Thus, the average-case (|(2)) competitive 

ratio is ( . - , ). The worst-case competitive ratio is ( d '"°" ). 

\ mm (J dj j J 1 \mmjjaijj 

Theorem 4. Extending model Ai to M2 with arbitrary producer request sizes, equal 
consumer capacities, arbitrary edge distances and allowing a single requests to be 
allocated across multiple consumers and running algorithm J{2, the expected total cost 
is E[G] = Y!, = i s{t) ■ d. 

Proof. The simplified algorithm J[2 is: 



Algorithm 3 Online Algorithm JH2 

1: Given: P ; C; c; dfj; s 

2: Initialize: €j = V/ e [n]; G = 0; n = \C\ 

3: for t = 1 to r do 

4: Choose P t e [m] independently and uniformly at random 
5: Choose an arbitrary request size s(t) = ^0, Z/ec( c 7 - t j)J 
6: for ^ = 1 to s(t) do 
7: C = {c; | ; 6 [n] Afj + s(t) < c] 

8: Choose 6 [n] independently and uniformly at random 

9: = ^ + 1 

10: G = G + dtj 
1 1 : end for 

12: end for 



By splitting the requests into unit sized blocks of size f D and using the fact that 
there is equal probability for the blocks to be allocated to any consumer, it follows from 
symmetry that a block of unit requests is equally likely to belong to any producer P,-. 
Using (20), (22) and (26) from Theorem 3 we get the expected cost of the edge selected 
in anay round as d. So the expected cost is E[G] = Y, r t =\ s (t) ' d and the average case 



competitive ratio is 
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Theorem 5. Extending model M2 to M3 with arbitrary producer demands, arbitrary 
consumer capacities and arbitrary edge distances and running algorithm J[2>, the ex- 
pected total cost is E[G] = YI t =\ s( f> " If' Vf - 

Proof. The simplified algorithm J[2> is, 



Algorithm 4 Online Algorithm 



Given: P; C; c; dij; s 

Initialize: £j = Vj 6 [«]; G = 0; n = \C\ 

for t = 1 to r do 

Choose P; E [ra] independently and uniformly at random 

Choose an arbitrary request size s(i) = |o, Zjec( c ; ~ ^j) 
for s = 1 to s(t) do 

C = jc ; | j 6 [n] A £j + s(t) < c} 

Choose Cj e n with probability ( ^„ C} c . j 

tj = tj + \ 
G = G + d fJ 
end for 
end for 



Let Xij be a indicator random variable that indicates that producer P, selects con- 
sumer Cj. As requests are split into unit sized blocks, the expected load on consumer 
Cj assigned by producer P, is equal to value of Xjj. Then E[Xjj] = E\<Cj | P,J = ^ - - . 
Let Yj be the indicator random variable for the load placed on consumer Cj in a specific 
round then^lY,] = Tip. E[X P . ,] • p(P ( ) = v Cj as each producer is equally likely to 
be selected in a round. Let Yj(f) denote the load placed on consumer Cj after r requests 
have been completed then E[Y:(t)] = Y r ,, s(t) ■ E[Y:] = ■ Y r ., s(t). Using unit 

sized requests the consumer load always remains proportional to its capacity. 

As the producers are chosen uniformly at random the probability that a consumer 
Cj was picked by a producer P, in any round is p[P t | Cj] = ^. Size of the request 
chosen (0, Tjjec( c j ~ at eac h iteration can be atmost equal to the remaining capacity 
available as it can be split amonst the consumers. 

Expected distance of the edge when consumer Cj is picked is, 

m 

E [ d 'j\ C j] = Tj d ''rP{Pi\Cj) (26) 
/=1 

= --y>,> (27) 

The expected cost of edge picked in a any round assuming equal failure times, 
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E[d u ]=Y j E[d U j\c J \-p(cj) (28) 



1 n m 

m j=i i=i ^j=i l ^ 

= vi " = — (30) 

2j j= i Cj m • n c 

Expected cost is E[G] = Yjf=\ s (0 ' ^ an d the average-case competitive ratio is 

My 

. Although the competitve ratio depends on the edge distances and consumer 

capacities, it is not possible to assume equal failure times using any other edge prob- 
ability although this online algorithm 3& is not optimal for the objective function in 

0- 
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Fig. 1. Comparision of Greedy and Randomized Vs. Optimal Offline for special cases 



The instances considered ranged from 1 to 100 producers and 1 to 100 consumers. 
The size of demands, edge distances and consumer capacities were picked randomly. 
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6 Conclusion 



The average-case competitive ratio of of y min d d j for the model with equal consumer 

capacities and arbitrary producer requests indicates that the performance depends on 
the quality of majority of links. For arbitrary consumer capacities with an average- 
case competitve ratio of { ^.J^. d . ; j the performance is decided by the quality of the 

links connected to the consumers with higher capacites. The optimal offline primal- 
dual algorithm runs in 0(r ■ n) time whereas the online algorithms take 0{r) time where 
r and n are the number of requests and consumers respectively. 
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